Simulation-Based Training: From
a Traditional Course to Remote Learning - the Covid19 Effect
Avraham Shtub1,
Matteo Kalchschmidt2 & Albachiara Boffelli2
1 Faculty
of Industrial Engineering & Management, Technion – Israel Institute of
Technology, Haifa, Israel
2 Department of Management,
Information and Production Engineering, University of Bergamo, Bergamo, Italy
Correspondence: Avraham Shtub, Faculty
of Industrial Engineering & Management, Technion – Israel Institute of
Technology, Room 507 Bloomfield Building, Haifa, Israel. Tel: +972-4-829-4454 E-mail: shtub@technion.ac.il
Simulation-Based Training: From a Traditional Course to Remote Learning -
the Covid19 Effect
The University of
Bergamo switched from regular classes to online classes due to the Covid19
pandemic during March 2020. In this paper, we focus on the lessons learned from
switching a project management course that combines traditional lectures with
Simulation-Based Training (SBT) to an online course with the same structure,
same curriculum and the same teaching team. Lessons learned are based on the
opportunity to compare the two methods of teaching and their learning outcomes.
Based on the analysis, conclusions about the future of this course and similar
courses are presented.
Keywords: online teaching, Covid19, project management, simulation-based training, traditional classes
1.1 Effective teaching in the COVID-19 era
The American
Association of Higher Education’s “Seven principles for good practice in
undergraduate education” (Chickering & Gamson, 1987) focus on key elements
that any training program should guarantee to achieve proper knowledge
development. These are:
1. Encourages
contacts between students and faculty.
2. Develops
reciprocity and cooperation among students.
3. Uses active
learning techniques.
4. Gives prompt
feedback.
5. Emphasizes time
on task.
6. Communicates
high expectations.
7. Respects
diverse talents and ways of learning.
The
recent lockdown due to the COVID-19 pandemic has dramatically challenged the
opportunity to respect such guidelines. Contacts among people have been
impossible limiting also cooperation among students. Providing prompt feedback
is more challenging since the synchronous interaction is more complex and not
as effective as through a personal and direct communication channel.
An
extensive debate on the role of online learning has been refurbished and
discussion about pros and cons and, more in general, of the learning
effectiveness of such a solution, has become dominant in several fields.
Starting
from these premises and from the personal experience of the authors during the
development of teaching programs in the lockdown period, this work aims to
provide evidence about the effects of online teaching perceived by university
students and the potential effects that online platforms can have on
Simulation-Based Training. The aim of this paper is, therefore, to present some
insights about the transition from a traditional to an online setting while
maintaining the same course structure, which includes simulation-based
training.
1.2 The project management course at the Department
of Management, Information and Production Engineering, Bergamo University
The
Project and Innovation Management course (from now on referred to as PIM) at
the University of Bergamo (Italy) is part of the master degree in Management
Engineering. Every year about 60 students, both Italian and international, are
enrolled in this course. The course load is equal to 12 ECTS (European Credit
Transfer and Accumulation System) and includes topics related to project
management, such as the project life-cycle, project organization, managerial
principles, planning and control methodologies, risk management, stakeholder
management; and to innovation management, such as taxonomy of innovations and
innovation dynamics. In terms of teaching methods, the course is structured
with 50% of the total number of teaching hours organized as lectures about the
theory, 30% as an application of the theory in case studies and analytical
applications, 10% as team activities to be developed typically after the
theoretical lectures and 10% as simulation-based training. The mixed structure
was thought to address the different needs of the students in terms of learning
styles, thus properly engaging them in the course. During the first lecture of
the course, all the relevant information about the course structure and
timetable, as well as the grading and evaluation are disclosed, to communicate
the high expectation, as this is one of the most important courses in terms of
workload and pedagogical impact. The different activities adopted during the
course development are aimed at reaching specific goals. The application
lectures are meant to encourage contacts between students and faculty and to
provide them with prompt feedback about their understanding of the topics. The
team activities are developed as an application of active learning techniques;
in fact, after having provided the students with the theory and having them
apply such theory in controlled situations (typically through case studies, as
well as by providing additional material), they are asked to apply them in
class during team activities. Finally, the simulation-based training is meant
to develop reciprocity and cooperation among students, as they are asked to
deliver a final assignment working in groups of five people. At the beginning
of the course, the students are free to choose whether to follow the course
full-time, following the previously-mentioned structure, or part-time, which
entails an individual final assignment and the exclusion from the
simulation-based training. As a result, only some of the students each year participate
in simulation-based training.
The
course evaluation is made of three components: a written exam, an oral exam and
the evaluation of the simulation-based training. The written exam (40% of the
final grade) includes multiple-choice questions, quantitative exercises and
small case studies. The oral exam (40% of the final grade) may include
questions both on the theoretical and the analytical part.
The
grading for the simulation-based training part of the course (20% of the final
grade) is based on two components:
·
Individual grades for
simulation exercises performed in class.
·
Teamwork grading of the final
assignment of the course.
The
final assignment is organized around the development of a project scenario
based on a real or fictitious project, analysis of the scenario, development of
a project plan and execution of the plan under uncertain conditions in a
simulated environment.
While
the course is usually taught in a face-to-face setting (from now on
“traditional setting”), due to the global pandemic, the course was taught in an
online setting during Spring 2020.
Simulation-based Training was integrated into the course due to its known
benefits (Henderson, C.,
Trotta, D. 2016, Strygacz, I., and
Shtub, A. 2018):
1) Emphasize questioning over answering on the part
of learners.
2) Provide opportunities to examine critically the
assumptions and implications that underline various decisions.
3) Expose the nature of problems and possible
solution paths.
4) Create an environment for learning that
generates discovery learning.
5) Promote skills in communicating, role-taking
problem solving, leading, and decision making.
6) Motivation and interest in a subject matter are
increased.
7) Evidence is offered for increased retention,
energizing the learning process and facilitating the understanding of
relationships between knowledge areas within a subject.
8) The focus of the simulation approach is on the
process of learning rather than on final results (e.g., preparations for taking
a test), and on representing the reality of a situation.
9) With the use of simulation-based training,
students can take risks without any unwanted or unfortunate consequences they
could face in the real world managing a real project.
In several studies, the successful use of a
simulator for teaching project management has been recorded (Davidovitch,
Parush, & Shtub, 2006, 2008, 2009; Perez, 2015). Working with a simulator provides the
opportunity for reflective observation. Particularly
in simulation-based training, the different scenarios and data items in the
simulator enable the conceptualization and integration of ideas and models considered
throughout the learning process. Moreover,
the simulator provides a continuous and dynamic environment for active experimentation
and is rapidly establishing itself as a critical factor influencing the
transfer of learning (Alessi, 1988).
Figure 1. Students' preferences for online and traditional
settings
Among the advantages of the online setting, students mentioned mostly
the possibility to get the recording (indicated by 22 respondents over 31),
time flexibility and management (stated by respectively 20 and 11 respondents),
avoiding the need for transport and travel (6 respondents), following lectures
from a comfortable environment (5 respondents) and improving notes and
understanding (4 respondents). From the other side, the main disadvantages were
the lack of interaction with the teacher (20 respondents) and with classmates
(17 respondents), technical issues (5 respondents), and the lack of motivation
and concentration (4 respondents). In the end, students were asked to indicate
whether they thought the University should switch back to a traditional setting
after the pandemic; interestingly only the 6% of respondents stated that they
would like to come back completely to the traditional teaching methodology,
meaning that the transition to the online setting was well managed and
generally appreciated by the students. Figure 2 reports the details of the
responses.
Figure 2. Details of the responses to the question "Do you
think that the university should switch back to the teaching methodology
used before Covid-19?"
Table 1.
Feedback from students that participated in the traditional course (Data
collected in June 2019)
|
|
Type of variable
|
Number of answers
|
% of answers < 6
|
% of answers ≥ 6
|
Average
|
D1 - The teachers
stimulate interest in the topics of the course
|
Likert scale 1-10
|
40
|
5.00%
|
95.00%
|
9.00
|
D2 - The teachers explain
the topics of the course clearly
|
Likert scale 1-10
|
40
|
2.50%
|
97.50%
|
8.83
|
D3 - I consider
applications and lab activities in the course useful for learning the topic
|
Likert scale 1-10
|
39
|
7.69%
|
92.31%
|
8.31
|
D4 - The teachers are
available for clarifications and explanations
|
Likert scale 1-10
|
44
|
0.00%
|
100.00%
|
8.50
|
D5 - I am interested in
the topics of the course
|
Likert scale 1-10
|
44
|
0.00%
|
100.00%
|
8.70
|
Table 2.
Feedback from students that participated in the online course (Data collected
in June 2020)
|
|
Type of variable
|
Number of answers
|
% of answers < 6
|
% of answers ≥ 6
|
Average
|
D1 - The teachers
stimulate interest in the topics of the course
|
Likert scale 1-10
|
31
|
3.00%
|
97.00%
|
8.84
|
D2 - The teachers explain
the topics of the course clearly
|
Likert scale 1-10
|
31
|
3.00%
|
97%
|
9.26
|
D3 - I consider
applications and lab activities in the course useful for learning the topic
|
Likert scale 1-10
|
31
|
0.00%
|
100.00%
|
8.52
|
D4 - The teachers are
available for clarifications and explanations
|
Likert scale 1-10
|
31
|
0.00%
|
100.00%
|
9.42
|
D5 - I am interested in
the topics of the course
|
Likert scale 1-10
|
31
|
0.00%
|
100.00%
|
8.55
|
The feedback collected about the simulation-based training was based on
the learning experience and the learning process. Concerning the learning
experience, the students were highly satisfied with the knowledge transferred
ex-ante (i.e., before the simulation-based lab) and with the effectiveness of
the online sessions. Still, some students were not completely sure about the
importance of the lab for the overall understanding. Finally, two groups of
students can be distinguished concerning the preference of traditional vs.
online setting: those who believed that the traditional setting would have
performed better, and those who were convinced about the higher effectiveness
of the online setting. Figure 3 reports the histograms of the results.
Figure 3. Histograms of the feedback about the learning
experience
Regarding the
learning process, we asked students to indicate how much the simulation-based
training contributed to their understanding of the different parts of the
course (i.e., to the overall objectives of the course). In particular, we were
able to assess that the simulation-based training was particularly helpful in
gaining a systemic view of the project, while it contributed slightly less in
transferring concepts related to the different modules of the course (namely,
scheduling, cost planning, risk planning, and resource management). In sum, we
can assume that, even in the online setting, the simulation-based training
proved to be effective in connecting different topics and enabling the students
to comprehend the overall picture. Figure 4 shows the histograms of the answers
about the learning process.
Figure 4. Histograms of the feedback about the learning process
2.4 Grades of the students
At last, we were able to compare the average grades (in the Italian
system assessed on a scale from 0 to 30, with 32 representing the summa cum
laude grade) achieved in the overall course and the final assignment of the
simulation-based training during the last 4 years. Among the last 4 years, from
2017 to 2019, both the course and the simulation-based lab were conducted in
the traditional setting, while only in 2020 both of them were conducted in the
online setting. From the graph reported in Figure 5, it is possible to notice
how the results were slightly higher in 2020, both for the overall course
(Final grade) and for the final assignment (Assignment evaluation), so we can
conclude that students were not penalized by the online setting. The ANOVA
conducted to test the difference among the means of
Figure 5. Final
grades of the students from 2017 to 2020
First of all, simulation-based online
training programs can indeed contribute positively to the learning experience.
Students provided a specific and explicit assessment of the direct positive effect
of this experience, but also the positive indirect effects this can have by
helping teachers become more effective and more supportive during the learning
process. The possibility to easily share with the teacher the specific issues
the student is facing or to ask a very punctual clarification and receiving
direct feedback has been extremely fruitful and valuable.
Second, students don’t show a unique
preference concerning traditional vs. online simulation training. We identify
that some students find value in developing online learning. We argue that this
is partially due to the novelty of applying an interactive lab through an
online platform, but partially also to the learning experience provided. On the
side, some students consider of higher value to have simulation training in a traditional
setting. Partially this is due to the contingent situation students faced
during the lockdown period, but we argue that this is due also to the specific
learning style students prefer. As Kolb (1981) suggests, different learning
styles exist and individuals are characterized by personal preference also
related to the specific background and the particular discipline considered.
Third, the development of the simulation
activity online has not harmed the effectiveness of the learning process. On the
contrary, the learning outcome (here measured by the final grades of the
students) shows no sign of decrease compared to previous years, claiming that
the online process is not less effective than a traditional setting.
Acknowledgments
All the authors contributed equally to the
development of the manuscript and agree on the final version submitted. The
authors would like to thank the students of the Project and Innovation
Management course (academic year 2019/2020) at the University of Bergamo for
their availability to provide feedback on the course, as well as the teaching
assistants that supported the development and administration of the course.
Alessi, S. M. (1988). Fidelity in the design of
instructional simulations. Journal
of Computer-Based Instruction, 15(2),
40–47.
Chickering, A., & Gamson, Z. (1987). Seven
principles for good practice in undergraduate education. American Association for Higher Education
Bulletin, 39(7), 3–7 (retrieved
July 15, 2003).
Davidovitch, L., Parush, A., & Shtub, A.
(2006). Simulation-based learning in engineering education: Performance and
transfer in learning project management. Journal
of Engineering Education, 95(4), 289–299.
https://doi.org/10.1002/j.2168-9830.2006.tb00904.x
Davidovitch, L., Parush, A., & Shtub, A.
(2008). Simulation-based learning: The learning-forgetting-relearning process
and impact of learning history. Computers
& Education, 50(3), 866–880.
doi>10.1016/j.compedu.2006.09.003
Davidovitch, L., Parush, A., & Shtub, A.
(2009). The impact of functional fidelity in simulator-based learning of
project management. International
Journal of Engineering Education, 25(2), 333–340.
Henderson, C., & Trotta, D. (2016). Infusing
simulation in classroom teaching. The
Journal for Research and Practice in College Teaching, 1(1), 1–5.
Kolb, D. A. (1981) 'Learning styles and
disciplinary differences'. in A. W. Chickering (ed.) The modern
American college, San Francisco:
Jossey-Bass.
Perez, M. (2015). A simulation
based optimization approach for stochastic resource constrained project
management with milestones. Thesis, Rochester
Institute of Technology.
Strygacz, I., &
Shtub, A. (2018). Combining simulation-based training and flipped classroom in project
management learning, Higher
Education Studies, 8(3), 85–93.